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Abstract 

This paper focuses on the research competencies of students in Instructional Design and Technology 
(IDT) programs. First, we describe how research is being taught to students enrolled in severed leading IDT 
programs. Next, we present findings from a survey administered to determine what knowledge and skills are 
being taught to IDT graduate students. Implications for the design of courses that focus on research are 
discussed. 



Introduction 

The competencies of professionals in the Instructional Design and Technology (IDT) field continue to 
receive attention. In recent years, books, articles, and conference presentations have focused on the 
competencies of instructional designers (Richey, Fields, & Foxon, 2001), training managers (Foxon, Richey, 
Roberts, & Spannaus, 2003), instructors in face-to-face and online settings (Goodyear, et. al. 2001; Klein, 
Spector, Grabowski, & de la Teja, 2004; Spector & de la Teja, 2001), and performance technologists (Fox & 
Klein, 2003). 

The current study continues a line of research conducted to identify the knowledge and skills that students 
in graduate IDT programs should obtain. An earlier study examined the optimal content and delivery method 
for a “foundations’' course in Educational Technology (Klein, Brinkerhoff, & Koroghlanian, 2003). Another 
study focused on the skills that IDT students should learn related to performance improvement (Klein & Fox, 
2003). The purpose of the current work is to answer the following two questions: 

• How is research being taught to IDT graduate students? 

• What research methods, processes, and issues are being taught to IDT graduate students? 

Method - Phase 1 

During this phase of the study, we obtained and analyzed the syllabi from research courses offered at 
several leading IDT programs to determine how re search is being taught to graduate students in the field. 

Participants 

Our sample consisted twelve IDT programs listed in the 2002 Educational Media and Technology 
Yearbook (EMTY). We only sampled programs that EMTY listed as offering a Ph.D. degree in Instructional 
Technology or Instructional Design and Development. The following institutions were included in our initial 
sample - Arizona State University, Florida State University, Indiana University, Pennsylvania State University, 
Purdue University, Syracuse University, University of Georgia, University of Memphis, University of Northern 
Colorado, Utah State University, and Wayne State University. 

Procedures 

We conducted a web search of each IDT program at the institutions listed above to determine their 
research course offerings. Next, we obtained the syllabi for research courses offered by each program. Some of 
the course syllabi were found on the web, while others were obtained by making direct contact with individual 
faculty members at each program. Syllabi for research courses offered at 10 or the 12 universities were obtained 
and include in our analysis. 

We then conducted a content analysis of the research course syllabi. Each syllabus was examined for 
the following: course title, credit hours, objectives, textbooks and other readings used, topics covered, 
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instructional activities, projects, and assignments. 



Results 

Several trends were identified by the content analysis. These trends are reported below. 

• Most of the IDT programs in the sample offer their own research courses. Several programs offer more 
than one research course. A few programs rely on others in their college to offer these courses. 

• Most of the courses examined focus on doing research. Many included requirements such as planning 
and conducting a research study. Some related to planning a dissertation study. Others focused on 
specific phases such as forming a research problem and conducting a literature search. 

• A range of quantitative and qualitative methodologies is taught in IDT research courses. 

• A wide variety of textbooks are used in IDT research courses. While several textbooks on research 
methodology have been adopted, preliminary analyses did not suggest any book as being most widely 
used. Several courses have students read excerpts from different textbooks. 

• A few courses require students to read, interpret, and analyze primary source documents such as 
published research articles. 



Method - Phase 2 

During this phase of the study, we developed and administered an online survey to determine what 
research methods, processes, and issues are taught to IDT graduate students. 

Participants 

We sent a request for participation to two division listservs owned by the Association of Educational 
Communications and Technology (Design & Development Division and Research & Theory Division) and to a 
listserv owned by the Professors of Instructional Design and Technology (PIDT). In addition, a request was sent 
to consulting editors of Educational Technology Research and Development and to a sample of individuals 
listed on the website, Who’s Who in Instructional Design and Technology. 

Our request led to 50 graduate students and 50 faculty members who completed the survey (N = 100). 
Respondents represented over 35 IDT programs mostly located in the United States. Three programs were 
located outside the U.S. (Australia, China, the Middle East). However, not all respondents listed their program 
affiliation when given the option. 

Survey Instrument 

The first section of the survey provided a lis t of 15 different research methods (see Table 1) and asked 
respondents to rate the degree to which students learn about each method using the following scale: 

0 = This research method is not covered in our curriculum. 

1 = Students are expected to acquire knowledge related to this method. 

2 = Students are expected to acquire skills related to this method. 

3 = Students are expected to acquire both knowledge & skills related to this method. 

The second section of the survey examined the degree to which students receive formal instruction on how 
to conduct a research study (see Table 2). Respondents used the following scale to rate ten research processes: 

0 = Students are not taught how to do this research process. 

1 = Students are taught how to do this process but never required to do it. 

2 = Students are taught how to do this process and do it for the first time during their thesis or 

dissertation. 

3 = Students are taught this process and do it before conducting their thesis or dissertation. 

The third section of the survey focused on five issues related to conducting research (see Table 3) and 
asked respondents to rate the degree to which students acquire knowledge and skills related to each issue using 
the same scale presented in section one. 

The fourth section of the survey asked respondents to identify whether they were a student or a faculty 
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member. Faculty were asked if they taught graduate-level research courses and if they supervised research 
theses and/or dissertations. An optional item asked respondents to provide the name of their university and 
program. 



Results 

Table 1 provides the rankings and mean scores for the 15 research methods included in the survey. 
These data show that five research methods (evaluation, qualitative, survey, experimental, & 
design/development research) received an overall rating between 2.06 - 2.32 indicating that most IDT students 
are expected to acquire skills related to these methods. Four other research methods (quasi-experimental, case 
study, mixed- methods, and descriptive research) received an overall rating of 1.76 - 1.94 suggesting that 
students in many programs are expected to obtain skills related to these methods. 

The research method data were analyzed to determine if any differences existed between students and 
faculty members. A significant difference was found for quasi-experimental methods [F (1,98) = 7.86, p < .01] 
and action research [F (1,98) = 14.29, p < .001]. In both cases, faculty members rated these methods higher 
than students. 

Table 2 shows that nine of the ten research processes included in the survey received an overall rating 
between 2.58 - 2.83 indicating that most IDT students are taught these processes and do them before conducting 
their thesis or dissertation. A significant difference was found for analyzing & interpreting research data [F 
(1,98) = 7.76, p < .01], writing research reports [F (1,98) = 10.56, p < .01], sampling participants [F (1,98) = 
8.46, p < .01], and developing data collection instruments [F (1,98) = 12.31, p < .001]. In all cases, faculty 
members rated these research processes higher than students. 

Table 3 provides the rankings and mean scores for the five research issues included in the survey. 

These data reveal that two issues (electronic searches & databases and analyzing research studies) received an 
overall rating between 2.43 and 2.36 respectively indicating that most IDT students are expected to acquire 
skills related to these issues. The three other issues (ethics, trends in IDT research, and professional contexts for 
research) received an overall rating above 1.50 suggesting that many IDT students are expected to acquire some 
knowledge about these topics. 



Table 1 - Rankings & Means for Research Methods 



Ranking 


Research Method 


Faculty 


Students 


Total 


1 


Evaluation 


2.54 


2.10 


2.32 


2 


Qualitative 


2.34 


1.96 


2.15 


3 


Survey research 


2.26 


2.02 


2.14 


4 


Experimental 


2.18 


1.98 


2.08 


5 


Design/Development research 


2.10 


2.02 


2.06 


6 


Quasi-experimental* 


2.22 


1.66 


1.94 


7 


Case study 


2.04 


1.58 


1.81 


8 


Mixed-methods research 


1.88 


1.72 


1.80 


9 


Descriptive research 


2.00 


1.52 


1.76 


10 


Action research* 


1.72 


.92 


1.32 


11 


Ethnography 


1.32 


1.08 


1.20 


12 


Meta-analysis 


1.18 


1.10 


1.14 


13 


Narrative research 


.88 


.92 


.90 


14 


Historical research 


.66 


1.02 


.84 


15 


Philosophical inquiry 


.64 


.74 


.69 



* A significant difference was found between faculty and students (p < .01). 
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Table 2 - Rankings & Means for Research Processes 



Ranking 


Research Processes 


Faculty 


Students 


Total 


1 


Specifying research questions 


2.94 


2.72 


2.83 


2 


Conducting a literature review 


2.94 


2.68 


2.81 


3 


Selecting a research topic 


2.86 


2.74 


2.80 


4 


Identifying research variables 


2.90 


2.64 


2.77 


5 


Constructing research hypothesis 


2.80 


2.66 


2.73 


6 


Analyzing & interpreting research data* 


2.90 


2.54 


2.72 


7 


Writing research reports* 


2.90 


2.46 


2.68 


8 


Selecting a research design 


2.76 


2.52 


2.64 


9 


Sampling participants* 


2.78 


2.38 


2.58 


10 


Developing data collection instruments* 


2.74 


2.14 


2.44 


* A significant difference was found between faculty and students ( p < .01). 



Table 3 - Rankings & Means for Research Issues 


Ranking 


Research Issue 


Faculty 


Students 


Total 


1 


Electronic searches and databases 


2.46 


2.40 


2.43 


2 


Analyzing research studies 


2.60 


2.12 


2.36 


3 


Ethics 


1.74 


1.78 


1.76 


4 


Trends in IDT research 


1.78 


1.64 


1.71 


5 


Professional contexts of research 


1.50 


1.56 


1.53 



Discussion 

The purpose of this study was to investigate the research competencies of graduate students in IDT 
programs by examining the skills and knowledge students acquire and how they attain them. The study was 
conducted in two phases. In Phase 1, we examined the syllabi from research courses offered at 10 leading IDT 
programs to determine how research is being taught to graduate students in the field. In Phase 2, we conducted 
an online survey to determine what research methods, processes, and issues are taught to IDT graduate students. 

Results from both phases of the study revealed that a range of quantitative and qualitative 
methodologies is being taught to students enrolled in IDT research courses. An interesting finding to emerge 
was that IDT faculty and students rated evaluation, qualitative, and survey methods somewhat higher than 
experimental and quasi-experimental methods. This finding can be explained by the applied nature of the IDT 
field. It is likely that evaluation, qualitative, and survey data collection techniques are covered in multiple 
courses when students are learning about needs assessment, working with subject matter experts, or conducting 
formative and summative evaluation of intervention. 

It is also possible that experimental and quasi-experimental methods may be receiving less attention in 
the IDT field than in previous years. This explanation is supported by a recent analysis of ETR&D (and its 
forerunners) that indicated the use of descriptive research methods has increased while the use of experimental 
methods has decreased in the last decade (Ross & Morrison, 2003). Other studies have suggested that consulting 
editors of ETR&D have a preference for applied research, case studies, and developmental methods over basic 
research studies (Klein, 1997). Furthermore, the use of developmental research methods has increased in recent 
years (Richey, Klein, & Nelson, 2003). 

Turning to research processes, we found most IDT students are taught the typical steps for how to 
conduct a research study and are required to do them before carrying out their thesis or dissertation. However, 
we did not find that any one particular textbook as being most widely used to teach students about these 
research processes. An interesting find to emerge was that students who responded to our survey indicated a 
lower agreement than faculty for four of the ten research processes - analyzing & interpreting research data, 
writing research reports, sampling participants, and developing data collection instruments. These differences 
suggests that IDT faculty who teach research courses should examine their objectives and activities to be sure 
these steps in the research process areas are being covered in enough detail. 
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During Phase 1 of the study, we were surprised to find that only few courses require students to read, 
interpret, and analyze primary source documents such as published research articles. However, the results from 
Phase 2 suggested that most IDT students are expected to acquire skills related analyzing research studies. It 
may be that our survey item was unclear and that respondents interpreted it to mean they are taught to analyze 
their own study (or their students’ studies in the case of faculty respondents). Regardless, our experience 
suggests that requiring students to read, interpret, and analyze published research articles is a robust 
instructional outcome and activity. 

We also think that students should learn about the professional contexts of research in the IDT field. 
However, our results suggest that this issue may not be covered in much depth. IDT is an empirical field; 
students who graduate from our programs should be able to apply research skills to a variety of contexts 
including business, industry, military, and school-based settings. 

Too often, research is thought of as the responsibility of academics that are required to do it to get 
tenure and promoted. Faculty who offer degrees in IDT should work to ensure that their graduates have 
competencies to be successful researchers. They should also push students to apply these competencies 
regardless of the setting in which they choose to work. 
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